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technical  report  on  a five-phase  program  to  develop  a 
Data  Management  System  and  a Student  Management  System. — Th i s report  reviews 
the  program  objectives,  program  plans,  products  developed;  and  summarizes 
subject-areas  for  significant  findings  or  developments.  ^These  Wall^  findings 
pertain  to  the  naval  air  training  system  features  that  impact  on  the  enhancement 
of  Individualized  student  pilot  training  treatment  and  on  student  training 

success  prediction  capabilities  and  options. ? 
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^ The  Data  Management  System  (DMS)  was  designed  and  implemented  on  to  a 
Nova  800  minicomputer  to  demonstrate  the  feasibility  of  operation.  This 
DMS  is  a user-oriented  online  system  which  permits  non-programmer  or  non- 
computer type  personnel  to  create  data  base  files,  update  these  files, 
search  single  or  multiple  files,  and  to  secure  hard  copy  printouts  of  search 
results.  A unique  feature  is  provided  which  permits  users  to  execute  cer- 
tain descriptive  and  analytic  statistics  on  records  located  by  the  search. a 
Users  are  assisted  in  operation  of  the  DMS  by  a wide  array  of  man-computep' 
dialog.  This  dialog  is  in  non-technical  conversational  1 anguagei__Jh«''UMS 
is  capable  of  being  implemented  and  operated__[n_a_wid#  -F-errge^BT^mil I i tary 
or  governmeat_in**»++etforTS' ^ Wtfere~thenee3  exists  for  turnkey  type  infor- 
mat ion  'management  systems. 
f 

S^The  Student  Management  System  consists  of  a description  and  specification 
of  the  concept,  products  and  records  required  to  enhance  an  individualized 
student  naval  pilot  training  treatment  program.  r"T\h is  system  uses  existing 
data  available  within  the  current  student  records/and  training  syllabus 
on  which  to  base  the  individualized  treatment.  The  heart  of  the  system  is 
the  use  of  products  which  more  quantitatively  reject  a student's  proficiency 
status  — weak,  strong,  and  average  performance  area?>\These  student  pro- 
files form  the  quantitative  bases  from  which  administrative  training 
personnel  can  select  individualized  treatment  options. 

References  to  past  reports  are  made  to  guide  the  jfeader  to  those  which 
treat  the  above  topics  in  more  detail.  ( N. 
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INTRODUCTION 


This  report  Is  the  final  technical  report  of  a five-phase  development 
program.  Phase  I began  on  May  16,  1972,  and  Phase  V was  completed  on 
November  15,  1976. 

The  program  objectives  were  to  develop  a Data  Management  System  (DMS) , 
and  a Student  Management  System  (SMS).  The  development  program  began  under 
Advanced  Development  Objective  43- 3 » Human  Factors  Technology,  and  was  shifted 
to  Technical  Development  Plan  **3-03X,  Education  and  Training.  The  initial 
development  agent  was  the  Naval  Air  Systems  Command;  it  was  later  transferred 
to  the  Navy  Personnel  Research  and  Development  Center.  Throughout  the  total 
development  program,  the  contracting  agent  was  the  Personnel  and  Training 
Research  Programs,  Psychology  Division,  Office  of  Naval  Research.  The 
development  effort  was  performed  on-site  at  the  Naval  Aerospace  Medical 
Research  Laboratory,  Pensacola,  Florida.  The  on-site  technical  monitors 
were  CDR  T.  Gallagher,  LCDR  N.  Lane  and  LCDR  R.  Doll. 

The  program  was  executed  by  Rowland  & Company,  Inc.,  Haddonfield,  New 
Jersey.  Dr.  George  Rowland  served  as  the  principal  investigator.  Mr.  Edward 
Marlowe  served  as  the  project  director  throughout  the  development  period. 

Mr.  Carlos  Escobar  served  as  the  computer  system  analyst  and  was  responsible 
for  the  design  and  implementation  of  the  DMS.  Various  other  Rowland  6 
Company  personnel  provided  valuable  support  to  the  results  achieved. 
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SECTION  II 


PROGRAM  OBJECTIVES 

The  objectives  of  this  development  program  were  twofold:  the  first  was 

the  development/demonstration  of  a prototype  Data  Management  System  that 
could  operate  on  a minicomputer;  the  second  was  the  development  of  a speci- 
fication for  a Student  Management  System. 

The  Data  Management  System  was  to  provide  for  a high  degree  of  automation 
associated  with  the  data  entry,  storage,  retrieval  and  analysis  of  large 
volumes  of  data  for  student  naval  aviator  and  student  naval  flight  officers. 
This  system  would  overcome  many  of  the  limitations  of  the  current  manual  and 
batch-oriented  data  processing  system.  The  DMS  would  be  used  to  handle 
student  selection  test  scores,  academic  and  flight  training  performance  data 
and  other  experimentally  derived  test  data  on  student  aptitude  capabilities. 

Such  a system  would  benefit  the  Aerospace  Medical  Research  Laboratory, 
which  has  a support  role  to  the  Bureau  of  Medicine  and  Surgery,  in  develop- 
ment of  selection  tests  and  criteria  used  for  screening  candidates  who  apply 
for  training.  The  Laboratory  also  has  a support  role  to  the  Chief  of  Naval 
Air  Training.  This  second  role  is  to  provide  predictions  on  students  who 
are  having  difficulty  in  the  training  programs.  These  predictions  provide 
a quantitative  score  which  is  an  estimate  of  the  student's  chances  of  com- 
pleting the  total  training  program  successfully.  The  DMS  would  provide 
support  to  the  continued  research  associated  with  these  predictions  and  with 
its  continued  daily  operation. 

The  Student  Management  System  is  intended  to  provide  the  concepts,  pro- 
cedures and  information  by  which  students  could  be  given  greater  individual- 
ized training  treatments.  To  individualize  training  treatments,  it  is  first 
necessary  to  differentiate  between  students.  The  student  Management  System 
concentrated  on  those  differences  that  existed  in  the  flight  performance 
domain.  A knowledge  of  the  strong,  medium  and  weak  flight  maneuvers  of 
students  at  all  positions  in  the  total  flight  syllabus  was  to  be  the  major 
emphasis.  Flight  Instructors  and  Training  Officers  would  use  this  knowledge 
to  adjust  the  student's  training  flight  hop  content  or  their  instructional 
techniques  to  improve  the  weak  areas  and  to  spend  less  time  on  the 
students'  strong  maneuvers. 

Concurrent  with  the  scheduled  development  of  the  DMS  and  SMS  was  the 
planned  development  of  the  Naval  Aviation  Training  Information  System  (NAT IS). 
Two  modules  within  the  planned  NATIS  were  the  Database  Module  and  the 
individualized  Pilot  Training  Subsystem.  The  DMS  was  considered  to  be  a 
viable  option  by  which  NATIS  could  fulfill  the  Database  Module  in  the  distri- 
bution network  of  NATIS.  The  SMS  would  fulfill  many  of  the  planned  require- 
ments of  the  Individualized  Pilot  Training  Subsystem  of  NATIS  (9,  10). 
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THE  DEVELOPMENT  PLAN 


The  development  concept  and  task  plan  are  presented  in  three  successive 
figures. 

The  development  concept  is  presented  in  Figure  3*1.  At  the  top  of  the 
illustration  is  depicted  six  process  elements  in  the  selection  and  training 
of  student  naval  aviators.  Information  on  the  students  is  available  from 
each  of  these  six  elements.  The  plan  called  for  the  successive  entry  and 
analysis  of  information  from  the  first  five  of  these  into  an  Experimental 
Data  Bank.  Computer  analyses  of  these  student  data  was  intended  to  provide 
an  emp i r i ca 1 basis  for  the  development  of  the  planned  Student  Management 
System  and  for  the  organization  of  the  data  files  which  were  to  be  handled 
by  the  planned  Data  Management  System.  Many  of  the  findings  which  are  re- 
lated to  the  Naval  Pilot  Training  System  were  based  on  these  empirical  data 
and  the  information  they  yielded  when  subjected  to  computer  analysis. 

The  task  structure  of  the  development  program  is  illustrated  in  Figure 
3.2.  Three  major  tasks  were  defined  and  pursued  as  the  means  by  which  the 
development  program  objectives  could  be  reached.  The  first  two  tasks 
parallel  the  two  products  to  be  developed.  The  third  task  covers  all 
effort  related  to  the  operation  of  the  Experimental  Data  Bank  and  other 
supporting  tasks. 

An  expansion  of  each  of  these  three  task  areas,  together  with  a time- 
phased  schedule  for  five  phases,  is  illustrated  in  Figure  3.3.  An  examina- 
tion of  the  tasks  under  each  of  the  five  phases  will  reveal  the  successive 
development  of  the  two  products  and  of  the  execution  of  supporting  tasks 
which  impact  on  the  planned  capabilities  of  the  Student  Management  System 
and  the  Data  Management  System  database  file  system  and  content.  No  attempt 
will  be  made  to  discuss  each  of  the  entries  in  detail  since  the  idea  of 
presenting  this  figure  was  to  depict  the  existence  of  a comprehensive  plan 
which  supported  the  development  program. 


APPROACH  OVERVIEW 


Figure  3-1 • Development  Concept 


Figure  3-2.  Program  Task  Organization  and  Outputs 
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Figure  3-3-  Task  and  Phase  Summary 
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DATA  MANAGEMENT  SYSTEM 


INTRODUCTION 

The  plan  for  this  product  called  for  the  successive  execution  of  a series 
of  steps  in  a planned  development  program. 

An  examination  of  the  task  and  phase  summary  for  the  Data  Management 
System  in  Figure  3-3  will  reveal  a progression  from  the  formulation  of  the 
initial  DMS  design  concept  through  several  design  steps  and  the  final 
implementation  and  demonstration  of  its  operation.  The  demonstrations 
were  completed  as  per  the  phase  schedule.  The  discussion  below  will  review 
the  progress  and  accomplishments  made  during  each  of  these  five  phases. 

PHASE  I - Operational  Concept 

What  kind  of  a system  would  provide  the  research-oriented  personnel  of 
the  Psychology  Department  with  a capability  to  rapidly  access  the  large 
volume  of  student  records  maintained  in  its  current  and  future  student  data 
banks?  What  kind  of  turn-around  time  did  these  personnel  need  in  order  to 
perform  timely  student  selection  research  and  to  update  the  Student  Success 
Prediction  System?  What  kind  of  new  data  elements  might  this  system  store 
and  make  accessible  in  order  to  improve  upon  the  selection  of  student  can- 
didates and  upon  the  training  success  prediction  of  student  naval  aviators 
and  of  student  naval  flight  officers?  What  kind  of  problems  were  present 
in  the  current  system  for  handling  the  large  volumes  of  student  selection 
and  training  performance  data?  These  were  some  of  the  concerns  in  the 
initial  study  phase.  Also,  how  can  this  data  handling  system  help  the  Chief 
of  Naval  Air  Training  to  improve  upon  its  management  of  students  as  they 
progress  through  their  lengthy  period  of  training? 

The  design  concept  which  emerged  was  influenced  by  several  technological 
and  political  factors  that  were  present.  The  political  factors  were:  (1) 

the  system  had  to  be  able  to  handle  not  only  the  student  data  for  the 
Psychology  Department,  but  the  system  per  se  had  to  have  some  benefit  to 
the  Chief  of  Naval  Air  Training;  (2)  the  system  could  not  be  placed  on  the 

current  UN  I VAC  4l8  computer  because  it  was  not  programmed  in  a high  level 

language  and  also  the  computer  was  an  early  generation  computer;  (3)  the 
Psychology  Department  was  in  the  process  of  obtaining  minicomputers  for 
their  laboratory  and  these  had  to  be  considered;  (4)  there  was  interest  on 
the  part  of  the  Navy  Personnel  Research  and  Training  Laboratory  that  the 
final  system  have  as  much  transportability  to  other  Navy  facilities  as 
practical  in  order  to  maximize  the  benefit  not  only  to  the  Psychology  De- 
partment, but  also  to  the  Navy  per  se. 

The  technical  factors  saw  the  emergence  of  the  minicomputer  as  a low-cost 
resource  which  used  high  level  language  (primarily  FORTRAN).  There  was  a 
development  to  make  these  minicomputers  serve  users  more  directly  than  large 
batch-oriented  computers.  Thus,  user-oriented  systems  were  emerging  and 
these  brought  the  full  resources  of  the  computers  directly  to  the  customer 
without  having  to  deal  with  a cadre  of  data  processing  personnel  in  order  to 
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' get  things  done.'  These  systems  were  sometimes  called  'turn-key'  because 
they  could  be  placed  into  operation  through  very  simple  procedures.  What  was 
lacking  with  these  minicomputers  was  the  software  programs  needed  to  support 
the  capability  to  handle  the  large  volumes  of  student  data  which  the  Psychology 
Department  needed  to  store,  access  and  manipulate  in  complex  combinations. 

The  design  concept  adopted  was  to  implement  a user-oriented  system  onto 
the  minicomputers  being  secured  by  the  Psychology  Department.  This  user- 
oriented  system  would  contain  all  of  the  software  programs  necessary  to: 

(I)  make  the  system  usable  by  dedicated  and  casual  users;  (2)  be  able  to 
handle  the  data  manipulations  required,  but  to  achieve  as  high  a degree  of 
automation  as  practical  so  as  to  reduce  the  dependency  upon  programmer  per- 
sonnel. To  keep  the  software  from  being  outdated,  it  was  decided  to  make  it 
generalized;  i.e.,  handle  data  manipulation  needed  to  store  records,  to  up- 
date these  records  frequently,  and  to  make  searches  of  the  records  so  as  to 
answer  any  type  of  operational  or  ad  hoc  questions  of  its  contents.  Such  a 
system  had  to  be  independent  of  the  specific  applications;  i.e.,  able  to  be 
used  in  a variety  of  operational  settings  and  serve  the  needs  of  users  in  each 
setting.  The  operational  concept  which  emerged  is  depicted  in  Figure  4.1. 

f 

The  system  depicted  shows  user  personnel  interacting  directly  with  the 
system  via  a terminal  display.  The  software  system,  termed  a Data  Manage- 
ment System,  provides  all  of  the  computer  programs  necessary  to  support  six 
types  of  user  transactions.  All  of  these  six  transactions  are  normal  to  the 
use  of  computer  resources  for  the  systematic  entry  and  effective  storage  of 
data  files  with  their  frequent  update  to  keep  the  contents  current,  and  a 
search  of  the  contents  to  retrieve  records  or  to  secure  answers  to  informa- 
tion queries  of  the  file  contents.  All  of  these  transactions  are  executed 
with  the  aid  of  the  man- computer  interactive  english  language  dialog.  The 
dialog  consists  of  instructions  which  guide  the  user  into  making  desired 
keyboard  entries  necessary  to  the  reliable  completion  of  the  transactions. 

A complete  description  of  the  Data  Management  System  appears  in  references 
(6)  and  (8). 

PHASE  II  - DMS  Requirements  Specifications 

The  design  of  the  DMS  provided  for  three  subsystems.  These  are:  (l) 

data  entry  subsystem;  (2)  data  base  subsystem;  and  (3)  query/retrieval  sub- 
system. The  development  of  subsystem  requirements  specifications,  the  com- 
puter programs  to  meet  these  specifications,  and  the  mechanization  of  these 
computer  programs  onto  the  NOVA  800  minicomputer  were  Dlanned  and  followed 
for  each  of  the  three  subsystems.  During  Phase  II,  the  requirements  were 
completed  for  the  Data  Entry  Subsystem  together  with  some  of  the  programming 
specifications.  No  implementation  onto  the  NOVA  computer  was  made  due  to  a 
delay  in  its  actual  arrival  in  the  Psychology  Department's  Laboratory. 

PHASE  III  - Continued  System  Specification  and  Initial  Implementation 

The  objectives  during  this  phase  were  to  continue  with  the  writing  of  the 
specifications  of  the  Data  Entry  Subsystem  and  the  initiation  of  the  specifi- 
cations for  the  Data  Base  Subsystem.  To  support  the  design  of  this  latter 
subsystem  a survey  of  existing  generalized  data  base  systems  was  conducted 
to  gain  an  overview  of  their  capability  and  design  features.  This  review 
eventually  led  to  the  adoption  of  the  relational  data  base  design  for  this 
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Each  information  system  area 
will  require  data  item  content 
be  placed  into  the  database 
that  is  specific  to  its  needs. 


Programmers  needed  to  write 
programs  to  support  appli- 


Figure  **.l.  Data  Management  System  used  to  Support  Information  Systems 
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subsystem.  The  relational  design  is  a relatively  new  development  in  terms 
of  the  number  which  have  been  implemented.  Because  of  its  simple  design,  the 
relational  concept  is  being  hailed  as  a breakthrough.  It  eliminates  unwieldy 
programming  which  has  increased  the  sheer  overhead  size  of  database  soft- 
ware packages.  The  relational  design  concept  is  described  in  detail  in 
reference  (14). 

PHASE  IV  - Implementation  and  Continued  Development 

Although  initial  programming  on  the  Data  Entry  Subsystem  began  in  Phase 
III  it  was  completed  during  this  phase.  Its  operation  was  tested  and  demon- 
strated to  the  customer. 

Implementation  of  the  Data  Management  System  onto  the  NOVA  800  mini- 
computer was  intended  to  demonstrate  feasibility  of  the  design  concept.  To 
provide  the  customer  with  an  operational  system,  an  effort  was  begun  to 
place  i t on  a UNIVAC  1 1 08  computer.  This  latter  computer  was  scheduled  to 
provide  the  future  ADP  support  for  the  Psychology  Department.  Implementation 
onto  the  UNIVAC  1 1 08  was  begun  with  the  Data  Entry  Subsystem.  Implementation 
of  the  Data  Base  Subsystem  was  begun  and  about  one-third  completed  during 
this  phase. 

PHASE  V - Completion  and  Demonstration 

This  phase  saw  the  completion  of  the  Data  Base  Subsystem  and  the  Query/ 
Retrieval  subsystem  onto  the  NOVA  800  minicomputer.  Effort  on  the  UNIVAC 
1108  was  discontinued  because  of  the  questionable  status  that  the  UNIVAC 
1 1 08  system  has  to  serve  as  the  ADP  operational  system. 

Detailed  descriptions  of  the  progress  made  during  each  of  these  five 
phases  are  contained  in  the  respective  phase  reports  (I,  II,  III,  IV,  V). 
References  to  quarterly  reports  for  each  of  these  are  contained  in  each 
phase  report. 

Three  documents  were  prepared  for  the  customer  to  support  the  use  of  the 
NOVA  800  based  Data  Management  System.  These  consisted  of:  (1)  a Users 

Manual,  (2)  a Data  Base  Administrator's  Manual,  and  (3)  a Design  Manual. 

These  are  covered  in  references  6,  7 and  8 respectively. 

SYSTEM  TRANSPORTABILITY 

One  of  the  criteria  for  the  development  of  the  Data  Management  System  was 
that  it  have  a high  degree  of  transportability.  This  was  achieved  by  having 
the  software  package  programmed  in  a high-level  language  FORTRAN  IV.  Approx- 
imately 97%  of  the  programs  are  so  programmed;  the  other  3%  are  in  assembly 
language. 

Use  on  other  computers  is  influenced  by  the  bit-configuration  and  the 
operating  system's  features.  The  NOVA  800  is  a 16-bit  computer,  which  is 
fairly  standard  for  other  minicomputers.  The  Data  Management  System  would 
operate  as  is  on  Data  General's  line  of  computers  including  the  ECLIPSE.  It 
would  also  operate  as  is  on  the  Digital  Control  Corporation's  D- 116  computer. 


STEPS  TO  ACCESS  DMS  AND  TO  SIGN  ON 


Enter  letters  DMS  via  terminal  keyboard 


To  Access  DMS 
And  to  SIGN-ON 


DATA  MANAGEMENT  SYSTEM  IS  READY  FOR  SIGN-ON 


DMS  Responds 


ENTER  YOUR  USER  NUMBER:  (number  is  placed  here) 

UrTfeyboardTn tries  are^tr^de  at  the  point  designated 
by  the  cursor  - for  this  and  all  other  keyboard  entries.) 


Start  SIGN-ON 


ENTER  YOUR  PASSWORD:  (password  is  placed  here) 


Complete  SIGN-ON 


i.  ■ 
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(Name  of  person)  SIGNED-CN  AT  (time  and  date) 

SELECT  AND  ENTER  TRANSACTION  FROM  THOSE  LISTED 
BELOW: 


FILE  MAINTENANCE 
CREATE  DATABASE 
UPDATE  DATABASE 
SEARCH  DATABASE 


WORKFILE  REPORT 
USER  CONTROL 
UTILITIES 
BATCH  UPDATE 


ENTER  THE  TRANSACTION  YOU  WISH  TO  PERFORM:  (name 
of  transaction  it  placed  here) 


DMS  Responds 
if  number  and  pass- 
word are  valid. 


Figure  4.2.  Sample  Man-Computer  Dialog 
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Conversion  to  other  minicomputers  would  require  about  four  to  six  man- 
months.  An  effort  is  in  progress  to  make  such  a conversion  to  the  Digital 
Equipment  Corporation's  PDP  11/70  minicomputer. 

SYSTEM  USAGE  IN  OTHER  APPLICATIONS 

Figure  4. 1 illustrates  the  design  and  operational  concept  of  the  Data 
Management  System.  The  design  incorporates  man-computer  dialog  by  which 
users  are  aided  and  directed  in  their  efforts  to  execute  the  data  manipula- 
tion transactions  provided.  This  type  of  design  has  been  described  as  user 
oriented  - in  contrast  to  programmer-oriented.  A user  may  be  any  person  in 
the  organization  who  has  a need  to  access  the  information  stored  in  the 
database  files.  Sample  man-computer  dialog  is  shown  in  Figure  4.2.  This 
dialog  covers  the  procedures  and  steps  by  which  a user  SIGNS-ON  and  selects 
a desired  information  transaction  type. 

Initially,  the  DMS  is  an  empty  system.  The  user  puts  into  it  the  type 
of  data  that  is  required  to  support  the  organization's  data  storage  and 
retrieval  information  needs.  The  type  of  information  will  vary  with  the 
type  of  organization  as  well  as  functions  it  performs.  Within  the  Chief 
of  Naval  Air  Training  (CNATRA)  complex,  the  DMS  has  a potential  to  serve  in 
many  situations.  This  was  recognized  in  the  formulation  of  the  Naval  Air 
Training  Command  Management  Information  System  (NATIS).  The  use  of  mini- 
computers was  projected  for  each  Training  Squadron  as  well  as  for  head- 
quarters use  (10).  This  recognition  lead  CNATRA  to  endorse  the  continued 
development  of  the  Data  Management  System  (9). 

The  DMS  can  be  used  to  computerize  the  files  of  the  Student  Management 
System.  It  will  be  recalled  that  a second  major  product  of  this  development 
plan  was  this  latter  system.  Its  features  and  potential  are  described  in 
another  section  of  this  report. 

The  need  for  and  role  of  Management  Information  System  in  Instructional 
Systems  Development  is  described  in  reference  15-  Based  on  our  understanding 
of  this  need,  the  Data  Management  System  would  be  ideally  suited  for  such  a 
role.  Those  personnel  who  are  responsible  for  this  area  should  examine  the 
potential  of  the  Data  Management  System  to  fulfill  this  role.  Not  only  does 
it  provide  the  capacity  to  store  and  organize  all  of  the  relevant  data  types 
associated  with  such  system  development  and  operation,  it  also  provides  the 
means  by  which  all  of  these  data  can  be  interrelated  for  their  information 
value.  Since  one  of  the  prime  reasons  cited  for  using  an  Information  Manage- 
ment System  is  to  provide  quantitative  information,  the  Data  Management 
System  has  the  unique  feature  that  this  quantitative  data  can  be  subjected 
to  statistical  analyses  within  the  Data  Management  System  proper. 

CAPABILITIES  AND  FEATURES  OF  THE  DMS 

What  can  the  DMS  provide  to  an  interested  user?  What  are  some  of  its 
design  features  that  make  it  of  particular  use  to  a wide  range  of  organiza- 
tions and  their  personnel?  Although  it  has  many  features,  the  most  sig- 
nificant of  these  are: 

• It  is  user-oriented  so  that  the  person  who  needs  to  use  its 
capabilities  can  do  so  directly  without  the  need  for  either 
programmer  or  computer  operational  personnel. 
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Its  design  includes  the  use  of  a database.  A database  is  a 
collection  of  data  files  that  are  organized  so  that  data 
items  within  the  collection  can  serve  the  information  needs 
of  all  organization  users.  The  organization  of  these  data 
files  is  intended  to  achieve  the  highest  possible  degree  of 
relatability  between  and  among  all  data  items  stored  in  this 
single  source  of  collected  data. 

It  provides  a large  and  unique  array  of  man-computer  dialogs 
that  are  programmed  in  non-techn ical  english-like  language.  This 
dialog  guides  the  user  in  the  proper  execution  of  the  selected 
information  tasks.  It  tells  the  user  what  type  of  an  error  was 
made  and  what  his  options  are  when  such  occur.  These  latter 
features  promote  a high  degree  of  reliable  man-computer  task 
completion  and  information  exchange. 

It  contains  within  one  integrated  package,  a total  system.  Many 
existing  Data  Base  Management  Systems  simply  provide  the  database 
part;  to  get  the  other  services,  you  must  purchase  additional 
packages  from  the  same  vendor,  or  from  some  other  vendor.  The 
DMS  provides  the  means  to  enter  data  for  storage  into  the  database, 
to  create  the  database,  to  search  the  database,  and  to  secure  re- 
ports. You  do  not  need  any  additional  software  to  implement  and 
to  operate  a database-oriented  information  management  system. 

The  user  interacts  with  the  DMS  via  a display  terminal.  All  of 
the  man-computer  dialog  appears  on  the  terminal  screen.  Once  the 
user  has  informed  the  DMS  of  the  information  task  he  wishes  to 
execute,  the  DMS  provides  the  step-by-step  instructions. 

The  FILE  MAINTENANCE  transaction  makes  it  possible  for  any  type  or 
number  of  record/files  to  be  entered,  stored,  and  updated.  Limited 
access  to  these  is  provided  since  this  is  not  considered  to  be  im- 
portant. The  record/files  entered  are  usually  those  which  are 
currently  being  maintained  manually  or  on  computer  files. 

The  CREATE  transaction  provides  a capability  to  organize  the  con- 
tents of  files  which  will  better  serve  the  needs  of  all  organization 
personnel . It  is  through  this  create  transaction  that  the  database 
Is  bui It. 

The  UPDATE  transaction  provides  the  means  by  which  the  contents  of 
the  database  can  be  kept  current.  This  can  be  done  both  in  an  on- 
line and  batch  mode. 

The  SEARCH  transaction  provides  the  means  by  which  the  contents  of 
the  database  can  be  queried.  It  provides  for  single-record  searches, 
mul t Iple- 1 tern  searches  and  even  a browsing  mode  where  you  can 
literally  go  from  record  to  record  within  a file  and  have  any  one 
or  all  of  them  printed  out.  There  is  the  option  of  what  or  of  how 
much  of  the  content  you  want  to  see  on  the  screen.  With  the  search 
transaction,  you  can  locate  all  records  that  meet  a combination  of 
information  types.  Up  to  99  data  item  combinations  can  be  speci- 
fied within  a search. 
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The  REPORT  and  STATISTICS  transactions  provide  the  means  by  which 
the  records  located  by  the  SEARCH  transaction  can  be  output  onto  a 
printer.  The  STATISTICS  transaction  permits  performance  of  statis- 
tical analysis  of  the  contents  of  the  records  located  by  the  SEARCH 
transaction.  The  results  of  the  statistical  analysis  can  be  viewed 
on  the  terminal  screen;  a hard  copy  of  these  results  is  available 
via  a printer  output. 

Through  a combination  of  the  CREATE,  SEARCH  AND  REPORT  transactions 
new  types  of  data  can  be  added  into  the  DMS,  related  to  the  data 
currently  stored  in  it,  treated  with  statistics  and  printed  outputs 
of  these  statistical  manipulations  secured. 

Through  the  USER  CONTROL  transaction,  you  can  control  those  personnel 
who  can  access  the  services  of  the  DMS,  control  the  databases  and 
files  to  which  they  can  have  access,  and  control  or  limit  access  to 
confidential  or  sensitive  information  in  the  database. 


SECTION  V 


STUDENT  MANAGEMENT  SYSTEM 


INTRODUCTION 

The  Student  Management  System  (SMS)  is  the  second  scheduled  product  of 
this  development  program.  It  is  a specific  application  of  the  more  general- 
ized Data  Management  System.  This  SMS  product  consists  of  a requi rements 
speci fi cat  ion  which,  when  implemented,  can  (a)  provide  a set  of  student 
handling  techniques  for  achieving  greater  student  individualization  of  train- 
ing, and  (b)  identify  the  training  system  data  elements  and  record  organiza- 
tion which  can  support  greater  individualization  and  provide  other  types  of 
management  information. 

The  student  handling  techniques  consist  of  a set  of  products  and  pro- 
cedures which  can  be  used  within  a squadron  training  environment.  The 
products  are  a family  of  both  individual  student  and  group  student  performance 
proficiency  profiles.  These  profiles  reveal  in  detail  and  in  quantitative 
form  the  weak,  average  and  strong  maneuver  skills  of  the  student.  With  this 
type  of  information  on  the  student,  both  the  student's  flight  instructor  and 
the  wing  leader  are  in  a more  favorable  position  to  plan  the  student's  sub- 
sequent hops.  Weak  areas  would  be  scheduled  for  greater  practice  and/or  a 
change  in  flight  instructional  techniques.  Strong  areas  would  receive  less 
practice  time  in  subsequent  hops.  Those  students  who  reveal  all  average  and 
above  average  manuever  skills  would  be  logical  candidates  for  hop  duration 
flight  time  reduction. 

The  discussion  below  will  describe  the  types  of  student  products  which  can 
be  made  available  from  the  existing  hop  grade  data,  the  squadron  personnel  who 
can  use  the  products,  and  a description  of  a student-oriented  record  concept 
that  will  support  the  generation  of  the  products  and  other  types  of  manage- 
ment information. 

FEEDBACK  CONCEPTS  AND  PRODUCTS 

To  control  any  complex  system,  it  is  necessary  that  regular  information  be 
made  available  to  those  personnel  who  are  responsible  for  its  control.  The 
control  i/iformation  must  be  relevant  to  the  resources  and/or  activities  which 
need  to  be  controlled  in  order  to  meet  the  system  goals  and  subgoals. 

Meeting  the  subgoal  of  greater  individualization  of  student  training  re- 
quires that  the  squadron  training  personnel  take  the  initiative  and  execute 
in  a daily  manner  all  of  the  required  procedures.  The  squadron  training 
personnel  involved  and  the  type  of  products  (information)  which  each  could 
use  is  illustrated  in  Figure  5.1. 

This  illustration  embodies  the  idea  of  feedback,  which  emanates  from  the 
area  of  cybernetics,  and  involves  the  study  of  automatic  control  systems. 
Automatic  control  of  a process  Is  achieved  by  the  continuous  monitoring  of 
the  process  performance  parameters  to  determine  the  in-tolerance  condition. 
This  monitoring  is  done  by  information  flow  to  those  who  have  a concern  for 
the  system  operation.  Based  on  the  'message'  contained  in  the  information 
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Figure  5-1-  Squadron  Feedback  Model  and  Product  Flows 
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flow,  training  personnel  take  the  appropriate  action  to  keep  the  system  moving 
toward  the  achievement  of  increased  individualization  of  student  training. 

The  type  of  message  which  each  person  in  the  training  organization  re- 
ceives is  tailored  to  his  duties  within  the  organization.  These  will  be 
discussed  in  more  detail  when  the  products  are  described.  But  at  a general 
level  of  discussion,  the  flight  instructor  would  receive  the  proficiency 
profiles  of  those  students  whom  he  is  currently  instructing.  Together  with 
the  student,  the  flight  instructor  would  then  plan  the  next  hop  maneuver 
content.  The  student  would  then  be  aware  of  his  weaker  areas  and  would  have 
an  opoortunity  to  brush  up  on  the  procedures,  etc.,  which  are  related  to  the 
maneuvers.  In  this  manner,  a partnership  is  established  to  achieve  a 
common  goa 1 . 

The  Wing  Leader  would  receive  the  profiles  of  all  of  the  students  in  his 
wing.  He  would  review  these  individually  so  that  he  would  be  aware  of  the 
nature  and  degree  of  deficiencies  revealed  by  the  profiles.  Where  common 
deficiencies  exist  among  many  students,  then  this  type  situation  could  be 
discussed  among  all  the  wing  instructors  to  explore  and  to  arrive  at  common 
types  of  remedial  instructional  techniques  that  may  be  applied. 

The  Wing  Leader  would  also  receive  student  profiles  at  hops  11  and  12  of 
the  presolo  stage;  such  profiles  are  termed  'outcome  profiles.1  These  pro- 
files would  reveal  the  student's  achieved  proficiency  at  the  end  of  his  stage 
training.  Where  the  deficiencies  still  existed,  the  Wing  Leader  would  be 
aware  that  the  remedial  instructional  techniques  were  not  effective  - with 
no  attempt  to  assign  the  underlying  cause.  This  situation  would  be  discussed 
again  among  all  of  the  wing  instructors  with  the  objective  of  sharing  those 
techniques  that  did  work  and  those  that  did  not.  Through  common  discussion, 
and  through  repeated  sessions,  the  most  effective  techniques  would  become 
evident.  When  this  happens,  both  the  student  and  the  squadron  personnel 
have  gained  and  the  subgoals  of  individualization  effectiveness  have  been 
promoted. 

The  squadron  training  officer's  span  of  responsibility  covers  several 
wings  and  also  extends  to  a total  phase  of  student  training.  Thus,  he  needs 
to  be  aware  of  the  operation  of  the  student  management  system  and  its  various 
products;  he  should  be  concerned  with  the  effectiveness  of  its  operation. 

In  Figure  5- 1 , effectiveness  data  would  be  covered  by  the  term  baseline  data/ 
prof i les. 

Baseline  data  would  cover  statistics  on  students  at  the  end  of  the  stages 
and  at  the  end  of  the  phase.  These  statistics  would  include  (a)  trend  informa- 
tion on  student  weak  areas  at  several  hop  positions  in  each  stage,  (b)  com- 
posite group  stage  student  maneuver  proficiency  profiles  at  regular  time 
intervals,  (c)  accumulative  flight-hours  statistics  of  students  for  each  stage 
and  grouped  into  three  student  grade  categories  - lower  third,  middle  third, 
and  upper  third,  (d)  student  flight  failure  statistics  which  reveal  the  hop, 
stage  and  maneuver (s)  failed,  and  (e)  trends  in  student  throughput  days  by 
stage  and  phase.  Each  of  these  types  of  information  will  not  be  discussed 
here  since  they  appear  in  the  yearly  program  progress  reports.  However,  as 
can  be  seen  from  the  types  of  information,  the  training  officer  would  be 
able  to  detect  in  quantitative  terms  the  utility  and/or  effectiveness  of  the 
individualization  program. 
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Two  other  squadron  management  personnel  are  shown  in  the  feedback  concept 
illustration.  These  are  the  Instructor  Training  Officer  and  the  Commanding 
Officer.  The  instructor  training  officer  should  have  a concern  about  the 
effectiveness  of  those  remedial  instruction  techniques  which  are  used  on  the 
students  who  have  weak  maneuver  areas.  This  information  can  be  similar  to 
that  which  the  training  officer  might  receive  and  include  firsthand  conver- 
sations with  the  flight  instructor  and  the  wing  leaders.  The  idea  here  is 
to  get  the  instructor  training  officer  involved  in  order  to  obtain  the  full 
benefit  of  his  own  and  his  flight  instructor  experience  with  varied  instruc- 
tional technique  effectiveness.  Where  the  statistics  reveal  little  or  no 
effectiveness  then  a major  goal  might  be  to  develop  new  instructional  tech- 
niques that  would  be  found  more  effective. 

Since  the  squadron  commanding  officer  has  the  final  responsibility  for  the 
effectiveness  of  the  individualization  program,  he  should  be  given  information 
about  its  frequency  of  application,  the  percentage  of  success,  and  the  impact 
on  stage/phase  training  flight  hours,  throughput  time,  attrition,  and  quality 
of  student  graduates. 

A summary  of  the  personnel  and  the  products  mentioned  above  is  tabulated 
in  Table  5- 1 . 

DESCRIPTION  OF  FEEDBACK  PRODUCTS 

A family  of  feedback  products  has  already  been  alluded  to  in  the  previous 
material.  This  section  will  give  a brief  description  of  each  product  to- 
gether with  an  indication  of  how  they  can  be  used  to  promote  the  goal  of 
increased  individualization  of  student  pilot  training. 

Individual  Student  Stage  Progress  Profile 

As  the  student  progresses  through  a flight  syllabus  stage,  he  receives 
individual  grades  on  each  of  the  scheduled  stage  maneuvers.  After  several 
hops  have  been  completed,  it  is  possible  to  compute  an  accumulative  grade 
value  for  each  of  the  maneuvers.  These  grade  values  can  then  be  plotted  on 
a graph  to  reveal  the  relative  proficiency  achieved  to  date  on  each  of  the 
maneuvers.  This  graph  is  termed  a profile.  A profile  has  the  merit  of 
ident i fyi ng  the  student 1 s weak  and  strong  maneuvers  as  well  as  the  number 
involved. 

Two  types  of  profile  have  been  developed.  The  first  is  based  on  a 4.0 
grade  system  and  the  second  is  based  on  a consistency  of  performance  idea 
and  uses  a percentage  value  computation. 

An  example  of  the  4.0  grade  system  profile  is  shown  in  Figure  5.2.  This 
profile  happens  to  be  on  a student  whose  cumulative  stage  grade  was  3.046, 
roughly  an  average  student.  When  th i s grade  value  is  taken  at  face  value,  it 
does  not  reveal  any  student  weak  or  strong  areas.  If  they  do  exist,  they  are 
obscured  by  the  fact  that  weak  areas  can  be  counterbalanced  by  strong  areas. 

In  the  case  of  the  student's  profile  shown,  this  is  exactly  what  happens. 

When  the  graph  is  reviewed,  it  can  readily  be  seen  that  the  student  had  4 or 
5 very  weak  areas  and  a similar  number  of  strong  areas.  Only  about  6 or  7 
of  the  maneuvers  fell  within  the  average  area. 


TABLE  5.1.  DATA  FEEDBACK  TYPES  AND  POSSIBLE  RECIPIENTS 


Headwork 

Basic  A i rwor k 
Procedures 
Prof  I i ght 
1 ax  i i ng 
Takeof  f 
Trans i t ion 
Slow  F I i tjht 
C I ean  S t a I Is 
Di  rty  Stalls 
Sp  i ns 
HAE 
LAE 

ipproach 

No  Flap  Landing 
Ful I Flap  Landin 
Stand  Field  Entr 
Home  Field  Entry 
Course  Rules 
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Figure  5-2.  Current  Grade  System  Student  Profile 


Now  let  us  examine  a second  student.  His  ^.0  grade  profile  is  shown  in 
Figure  5.3.  No  weak  areas  or  strong  areas  are  apparent.  Yet  the  student's 
cumulative  flight  grade  was  3.062  - nearly  identical  to  the  previous  student. 
From  these  two  profiles,  it  can  readily  be  seen  that  the  two  students  are  not 
equally  proficient;  also,  it  is  quite  evident  which  one  could  benefit  from 
some  positive  individualized  training.  Thus,  the  profile  technique  not  only 
overcomes  the  limitations  of  using  the  cumulative  hop  grade  for  progress 
evaluation  but  it  depicts  in  quantitative  manner  the  weak  and  strong  areas 
of  the  student's  syllabus  maneuvers. 

Student  Consistency  Performance  Profiles 

Within  a particular  stage  of  flight  training,  a student  is  required  to 
practice/demonstrate  his  degree  of  proficiency  many  times.  For  each  hop  he 
is  assigned  a specific  proficiency  grade;  usually  an  average,  above  average, 
or  below  average  for  each  of  the  maneuvers.  If  we  accept  the  assumption  that 
the  student  who  receives  a greater  percentage  of  average  and  above  grades  is 
more  qualified  than  one  who  receives  some  below  average  grades  on  any  one 
maneuver  or  on  all  maneuvers,  then  it  is  possible  to  express  these  student 
differences  in  a quantitative  form.  This  was  done  for  the  same  students  on 
which  i*.0  grade  value  profiles  were  made  and  which  were  illustrated  in  the 
two  previous  figures.  These  two  consistency  profiles  are  shown  in  Figures 
5.**  and  5-5. 

Consistency  values  are  computed  by  adding  the  total  number  of  average  and 
above  average  grades  received  and  then  dividing  by  the  total  number  of  grades 
received.  A value  of  1.00  indicates  that  all  of  the  student's  grades  were 
average  or  above.  A value  less  than  1.00  indicates  that  some  below  average 
grades  were  received.  A 0.50  value  would  indicate  that  at  least  one  half  of 
the  grades  received  were  of  the  below  average  category.  Thus,  faced  with  a 
prediction  of  what  the  student  proficiency  level  is  likely  to  be  on  the  next 
hop,  this  value  would  be  the  best  estimate. 

In  the  case  of  the  two  consistency  profiles  illustrated,  the  individual 
level  of  maneuver  consistency  achieved  by  each  student  is  quite  apparent  as 
well  as  the  dramatic  differences  between  these  two  students.  Although  the  **.0 
grade  profile  does  reveal  a student's  weak  areas,  the  consistency  profile 
expresses  these  weak  areas  in  a form  which  is  easily  understood  and  whose 
consequences  can  also  be  readily  judged  both  in  terms  of  current  proficiency 
and  next  hop  estimates.  Thus,  the  consistency  profiles  reflect  a different 
aspect  of  student  performance  proficiency  than  either  a simple  hop  grade 
value  or  the  *».0  grade  maneuver  profile.  For  instance,  the  student  who  re- 
veals a 0.500  consistency  value  for  HAE/PPEL  at  hop  11  in  the  presolo  stage 
might  be  considered  a high  risk  for  soloing!  The  Wing  Leader,  the  Training 
Officer  and  even  the  Standards  Officer  would  want  to  be  apprised  of  this 
student's  situation!  (Student  in  Figure  5.*0 

Group  Student  Profiles 

These  profiles  are  similar  to  individual  student  profiles  but  their  grade 
values  or  consistency  values  are  based  on  the  summation  of  a given  number  of 
students'  individual  grades.  These  profiles  are  capable  of  revealing  to  the 
Wing  Leader/Training  Officer  consistent  weak,  average  and  strong  maneuver  areas. 
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Figure  5-3-  Current  Grade  System  Student  Profile 
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Figure  S-^-  Consistency  Profile  of  Student 
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Figure  5-5-  Consistency  Profile  of  Student 
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Where  these  exist,  some  type  of  total  squadron  effort  would  be  appropriate 
since  they  would  mean  that  the  existing  instructional  techniques  are  not  as 
effective  as  they  could  be. 

The  group  profiles  at  the  end  of  the  stage  would  depict  the  quality  of 
student  graduates.  Like  the  individual  student  profile,  the  composite  grad- 
uate profile  would  reveal  similar  information. 

To  track  the  benefits  that  might  accrue  from  the  application  of  an  indiv- 
idualized training  program,  the  examination  of  stage  graduate  profiles  before 
and  after  the  program  had  been  placed  into  effect  would  identify  the  areas 
of  effectiveness  and  non-effectiveness. 

Baseline  Statistics  and  Impact  Trends 

The  Idea  of  these  statistics  is  to  quantify  the  effects  of  the  individ- 
ualized training  treatments  of  student.  It  can  be  logically  expected  that  if 
the  students  and  training  system  are  benefiting,  some  quantitative  evidence 
should  be  available. 

The  benefits  should  reveal  themselves  in  trend  type  information.  The 
following  types  of  trend  information  should  be  made  available: 

a.  Changes  in  the  magnitude  of  Individual  and  composite  student 
weak  maneuver  areas. 

b.  Decreases  in  the  magnitude  and  number  of  composite  student 
weak  areas  at  the  end  of  each  stage. 

c.  Decreases  in  the  flight  syllabus  stage  failures  where  high 
failure  rates  were  previously  in  evidence. 

d.  Where  some  students  are  accelerated  because  they  have  no 
weak  areas,  a reduction  in  total  flight  hours  per  stage 
should  accrue.  Again  this  type  of  impact  will  show  up  only 
when  many  students  have  in  fact  been  accelerated. 

e.  Student  throughput  days  statistics.  At  this  time,  there 
would  be  no  basis  for  believing  that  throughput  days  of  train- 
ing would  be  reduced  by  the  above  types  of  student  treatments. 

Flight  hours  could  be  expected  to  be  reduced  but  other  factors 
would  more  likely  determine  throughput  days.  If  the  profiles 
were  used  as  a basis  for  reducing  the  number  of  stage  hops 
Instead  of  reducing  the  duration  of  flight  hops  then  some  im- 
pact on  throughput  days  could  be  expected. 

RECORD  CONCEPT 

Objective 

The  student  naval  aviator  record  concept  being  proposed  herein  is  intended 
to  fulfill  several  distinct  objectives.  The  first  objective  is  to  build  upon 
the  record  types  now  contained  In  the  student  aviation  training  jacket.  This 
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record  concept  evolved  after  many  years  of  experience  and  its  current  utility 
is  not  going  to  be  ignored.  The  second  objective  is  to  build  a record  con- 
cept that  will  support  a continued  program  of  analysis  which  can  help  train- 
ing administrators  to  detect  less  effective  training  operations  and  to  im- 
prove upon  the  training  system,  thereby  achieving  greater  efficiencies  in 
its  operation.  The  thi rd  object i ve  is  to  provide  the  means  by  which  greater 
student  individualization  of  training  treatments  can  be  implemented  at  the 
squadron  level.  Individual izat ion  of  training  is  used  here  to  denote  train- 
ing treatments  to  meet  each  student's  needs  to  a greater  degree  than  current 
practice  permits.  A fourth  objective  is  to  formulate  a record  concept  that 
will  represent  in  data  form,  the  physical  analog  of  a student's  administra- 
tive and  training  events  and  which  will  permit  a reconstruction  of  these 
events  and  place  them  into  their  proper  time  position  within  the  total  time- 
span  of  the  student's  training  period.  This  objective  is  considered  impor- 
tant because  of  the  fundamental  role  that  the  time-domain  has  in  student 
skill,  and  knowledge  acquisition  has  in  training  progress  and  outcomes. 

Record  Structure 


The  record  structure  which  is  envisioned  at  this  time  and  which  will 
fulfill  the  objective  cited  above  would  consist  of  six  separate  record 
types.  These  records  are  as  follows:  (1)  administrative,  (2)  training 

summary,  (3)  phase  summary,  (A)  stage,  (5)  flight  hop,  and  (6)  academic. 

There  is  a continuous  functional  relationship  between  the  flight  hop  record/ 
academic  record  with  the  respective  stage,  phase  and  training  summary  records. 
The  functional  relationship  is  depicted  in  Figure  5-6. 

As  illustrated,  when  certain  data  are  input  into  the  ground  school  and 
in  the  hop  records,  these  data  are  automatical ly  collated  and  an  entry  made 
in  the  stage  record.  For  instance,  when  a student  completes  all  of  the  re- 
quired hops  in  a stage,  the  daily  hop  grades  which  appear  in  the  hop  record 
are  summated,  a A.O  grade  value  computed  and  its  outcome  is  then  placed  in 
the  stage  record.  A host  of  similar  functional  relationships  exist  in  the 
proposed  structure.  These  will  be  discussed  in  more  detail  in  the  material 
that  follows.  Similar  functional  relationships  (dependencies)  exist  between 
the  stage  record  and  the  phase  record  and  between  the  phase  record  and  the 
training  summary  record.  As  the  student  completes  his  daily  hops,  each  of 
the  records  is  automatically  updated  to  reflect  his  new  status.  Thus,  each 
record  in  the  structure,  when  accessed,  would  reflect  his  current  training 
syllabus  status  and  his  current  performance  proficiency  status.  For  con- 
venience of  reference,  the  record  structure  has  been  designated  as  being 
composed  of  five  levels;  these  are  identified  in  Figure  5.6  as  running  from 
01  to  05-  The  record  will  also  signify  that  each  level  of  records  and  their 
respective  contents  is  of  interest  to  certain  student  management  personnel 
In  the  chain-of-command  from  flight  instructor  to  CNATRA  operations  staff 
per sonne l . 


05  Level 


04  Level 


03  Level 


02  Level 


01  Level 
01  Level 


Figure  5.6.  Student  Management  System  Student  Naval  Aviator 
Record  Structure  Concept 
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Record  Content  and  Utility 


The  content  of  the  records  described  above  was  specified  so  as  to  support 
the  four  objectives  cited  plus  the  recognition  that,  within  a training  organ- 
ization, different  management  personnel  have  need  for  different  types  of  data 
and  information  to  help  them  fulfill  their  job  tasks.  In  the  proposed  struc- 
ture, the  01  and  02  levels  would  be  of  interest  to  flight  instructors  and  the 
training  officer.  These  are  the  data/information  that  they  work  with  on  a 
daily  basis  to  manage  the  students  effectively  and  efficiently.  The  03  record 
level  is  of  interest  to  squadron  commanders,  training  officers  and  TRAWING 
command  and  staff  personnel.  The  content  will  tell  them  how  well  the  train- 
ing goals  of  schedule,  rate  and  quality  control  have  been  met.  This  is  after 
the  fact.  If  they  want  to  know  the  status  of  these  goals  during  the  stage 
training,  then  they  would  want  to  access  02  level  data  as  it  is  being  accum- 
ulated. Thus,  if  a large  group  of  students  is  behind  schedule,  they  would 
be  aware  of  this  information  and  could  take  what  action  is  desirable  and 
feasible. 

The  content  of  03  level  records  (phase)  would  be  of  special  interest  to 
TRAWING  operations  and  plans  personnel  since  current  TRAWING  operations  in- 
clude the  complete  training  of  at  least  two  phases,  the  basic  and  the  advanced 
phases.  The  phase  record  would  reveal  statistics  on  (1)  the  realized  Pilot 
Training  Rates,  (2)  the  schedule  position  of  students  who  completed  the  phase, 
(3)  the  factors  (such  as  weather,  aircraft  availability,  instructor  avail- 
ability, and  student  avai labi 1 i ty)  which  determined  the  individual  student's 
schedule  position,  (4)  the  quality  level  of  the  students  who  completed  the 
phase  - expressed  in  terms  of  grade  distributions,  grade  means  and  standard 
deviation,  or  the  new  grade  concepts  developed  by  Rowland  6 Company.  These 
include  the  selected  maneuver  grades/profiles  and  the  maneuver  reliability 
grade/profile  options. 

The  04  record  (training  summary)  content  is  primarily  factual  documenta- 
tion of  what  parts  of  the  syllabus  a particular  student  completed.  It  tells 
what  pipeline  the  student  was  assigned  to,  the  aircraft  models  and  flight 
hours  flown,  how  much  instruction  training  was  received  in  actual  flight  and 
in-flight  simulators,  what  grade  values  (expressed  as  standard  scores)  were 
received  in  ground  school,  in-flight  syllabus  and  the  final  overall  grade. 

In  addition,  it  contains  data  on  carrier  qualification,  student  accidents, 
student  pilot  disposition  boards,  and  other  descriptor  information.  Rowland 
6 Company  is  also  reflecting  information  pertaining  to  the  students'  (a) 
final  throughput  time  in  relation  to  standard  syllabus  time,  (b)  final  flight 
hours  in  relation  to  standard  syllabus  flight  hours,  (c)  maneuver  grade  and 
the  student  performance  reliability  grade. 

The  05  record  (administrative)  content  is  primarily  for  unique  identifying 
data  about  the  individual  student.  This  consists  of  traditional  items  such 
as  name,  social  security  number,  branch  of  service,  rank,  date  of  rank,  etc. 
The  content  of  this  record  may  well  follow  that  being  developed  for  the 
Personnel  Record  of  the  planned  NAT  IS.  Rowland  & Company  does  not  have  any 
special  content  recommendations  for  this  record.  Its  primary  use  is  to  keep 
Navy  administrative  information  separate  from  the  CNATRA  training  function 
records . 


RECORD  DESCRIPTIONS 


This  section  will  describe  each  of  the  six  record  types  already  intro- 
duced. The  specific  content  data  element  type  and  the  quantitative  value 
of  each  will  be  indicated  where  possible.  The  source  of  the  data  which 
goes  into  each  record  will  be  identified;  this  will  be  done  especially  for 
those  data  types  which  tie  one  record  into  another  in  terms  of  dependency. 
Since  a cumulative  dependency  exists  from  the  top  record  downward,  it  is 
planned  to  discuss  the  records  at  the  lowest  level  first  so  that  the  proper 
data  elements  will  have  been  introduced  in  z record  which  feeds  upward  into 
the  higher  level  records. 

Hop  Record 

The  hop  record  would  contain  all  of  the  information  now  contained  on 
ATF  type  forms.  For  discussion  purposes,  it  has  been  separated  into 
(a)  administrative,  (b)  training  syllabus,  (c)  hop  description,  and  (d) 
maneuver/grades  of  that  hop. 

The  administrative  data  would  consist  of  identifying  data  such  as 
student  SSNl,  and  NAME. 

The  training  hop  identification  would  consist  of  such  data  as  (a)  syl- 
labus hop  number  designation,  (b)  date  of  hop,  (c)  instructor  identification 
number,  (d)  hop  duration,  (e)  type  of  hop  - solo  or  dual,  (f)  hop  type  in 
terms  of  regular,  extra  time,  special  check;  and  (g)  hop  outcome  in  terms 
of  up  or  down. 

The  maneuver /grade  data  would  consist  of  an  itemization  of  each  hop 
maneuver  type  demonstrated,  practiced  and/or  graded  - together  with  the 
grade  assigned  to  each,  in  addition,  the  grade  for  that  hop  would  be 
shown. 

The  cumulative  maneuver  grade  data  would  consist  of  a running  total  of 
(a)  the  number  of  unsat  grades,  (b)  the  number  of  BA  grades,  (c)  the  number 
of  A grades,  and  (d)  the  number  of  AA  grades. 

A hop  record  would  exist  for  each  hop  flown.  Access  to  a particular 
hop  record  would  be  via  the  student's  SSN  and  syllabus  hop  designation 
number.  The  format  for  the  display  of  the  hop  record  data  on  the  terminal 
screen  is  not  a topic  of  this  working  paper.  The  format  would  have  to  be 
coordinated  with  user  type  squadron  personnel. 

A listing  of  the  above  data  elements  is  shown  in  Figure  5-7. 

Stage  Record 

The  stage  record  would  be  a running  and  final  summary  of  all  training 
events  in  each  stage  of  training.  Depending  upon  pipeline,  there  are  approx- 
imately 20-25  stages.  Thus,  each  stage  record  would  have  a unique  identifier. 
These  identifiers  would  correspond  to  those  already  being  used  in  the 
syl labus  file. 


HOP  RECORD 


Identification 

Name  of  Student 
APL  Number 
Training  Stage 
Stage  Hop  Number 

Admi ni strat i ve 

Hop  Date  - M/Y 

Hop  Duration  - hrs/tenths 

Hop  Type  - regular,  special,  etc. 

Hop  Up  or  Down 

Solo  or  Copi lot 

Aircraft  Type 

Instructor  Number 

Hop  Grade  Data 

Number  of  unsatisfactory  grades 
Number  of  below  average  grades 
Number  of  average  grades 
Number  of  above  average  grades 

Cumulative  Hop  Grade 

Cumulative  unsat  grades 
Cumulative  below  average  grades 
Cumulative  average  grades 
Cumulative  above  average  grades 

Maneuver  Grade  Data 

Maneuver  identification 
Maneuver  grade 
Maneuver  identification 
Maneuver  grade 
etc. 


Figure  5-7.  Mop  Record 


STAGE  RECORD 


Identification 

Name  of  Student 
APL  or  SSN  of  Student 
Syllabus  Stage 

Administrative 
Squadron  ID 
Date  Started 
Date  Completed 
Aircraft  Model 


Schedule  Position  Data 

Projected  completion  date 
Actual  completion  date 
Hop-days  lost: 

. Weather 

. Aircraft  availability 
. Instructor  availability 
. Student  availability 
. Other 


Cumulative  Stage  Grade  Data 
. Number  of  unsat 

Number  of  below  averages 
. Number  of  averages 
. Number  of  above  averages 


Maneuver  Profile  Data 

. Reliability  value  for  each  stage  maneuver 
. Cumulative  grade  for  each  stage  maneuver 


FI ight  Hours 

. First  Pilot  - standard 
. First  Pilot  - actual 
. Copilot  - standard 
. Copilot  - actual 


Figure  5*8-  Stage  Record 


The  stage  record  would  contain  (1)  administrative  data,  (2)  training 
squadron  data,  (3)  schedule  position  data,  (4)  stage  maneuver  profile  data, 
(5)  cumulative  flight  time,  and  (6)  cumulative  raw  grade  data. 

The  administrative  data  would  consist  of  the  student's  name,  social 
security  number,  pipeline  assignment  and  stage  identification. 

The  training  squadron  data  would  consist  of  squadron  identification, 
date  student  reported  in,  date  student  reported  out  and  aircraft  model  be* 
ing  used  in  this  stage. 

The  syllabus  schedule  position  data  would  be  a running  summary  of  the 
current  student's  schedule  position.  The  position  would  be  expressed  in 
terms  of  the  number  of  hops  - behind,  ahead  or  0 for  on-schedule.  When  a 
hop  is  lost,  the  reason  for  the  loss  would  be  recorded  as  (a)  weather, 

(b)  aircraft  availability,  (c)  instructor  availability,  or  (d)  student  avail- 
ability. The  information  from  this  record  would  serve  as  inputs  into  the 
scheduling  officer's  data  needs. 

The  cumulative  stage  profile  data  would  consist  of  computed  data  ex- 
pressed either  as  *t.O  grade  value  or  in  reliability  terms  and  would  reveal 
the  student's  current  proficiency  on  each  of  the  stage  maneuvers.  Being  a 
cumulative  profile,  the  grade  data  from  each  stage  hop  would  be  used  to 
compute  the  data  values  for  the  profiles.  The  profiles  would  be  available 
to  the  flight  instructor  or  training  office  either  as  displayed  on  the 
terminal  or  as  a printed  output. 

The  cumulative  stage  grade  data  would  still  be  available  to  flight 
instructors,  training  officers  and  student  disposition  boards.  These  data 
would  consist  of  the  cumulative  number  of  (1)  unsat  grades,  (2)  below 
average  grades,  (3)  average  grades,  and  (4)  above-average  grades.  In 
addition  to  these,  a cumulative  stage  grade  value  would  be  automatically 
computed  from  the  above  and  the  value  stored  in  the  record.  This  value 
would  be  updated  after  each  hop.  This  same  value  would  then  be  used  as 
the  stage  grade  when  the  stage  hops  had  been  completed. 

The  cumulative  flight  time  hours  data  would  consist  of  (1)  first  pilot 
hours,  (2)  co-pilot  hours,  and  (3)  number  of  hops  flown.  These  data  would 
serve  as  inputs  into  the  phase  record  where  similar  type  data  are  being 
accumulated. 

A listing  of  the  above  data  elements  within  a record  layout  is  shown 
in  Figure  5”8. 
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How  can  training  administrators  use  the  data  placed  into  the  stage  record 
to  achieve  training  system  goals  and  achieve  individualized  student  training 
treatment  goals? 

Training  system  goals  are  the  achievement  of  a training  schedule  for  each 
student  so  that  a projected  training  rate  can  be  met  and  a quality  student 
product  achieved  that  has  a low  risk  of  future  failure.  As  the  student  pro- 
gresses from  one  stage  to  another  the  information  about  the  student's  schedule 
position  is  available.  If  a group  of  students  are  behind  schedule  at  the  end 
of  the  2nd  stage  of  training,  then  this  information  can  be  used  to  give 
priority  to  their  scheduling  in  the  next  stage  or  stages  in  order  to  over- 
come their  behind-schedule  position.  In  this  way,  the  projected  training  rate 
is  more  likely  to  be  met. 

Each  stage  record  contains  data  which  can  be  used  to  secure  a set  of 
syllabus  maneuver  profiles  for  each  of  the  stages  completed  in  the  phase. 

The  profiles  reveal  the  relative  strength  and/or  weakness  of  the  student  on 
all  maneuver  types  and  may  be  used  to  detect  the  presence  of  consistent  weak 
areas  across  all  stages.  For  instance,  a student  who  blew  emergency  proce- 
dures at  all  stages  of  his  training  would  be  a higher  risk  student  than  one 
who  had  consistently  average  or  above  average  grades.  Such  a student  could 
be  given  extra-hops  to  help  him  overcome  this  area  rather  than  send  him  on 
to  the  next  stage/phase  with  known  weaknesses.  The  reliability  value  for 
each  stage  maneuver  is  useful  for  the  detection  of  these  weak  areas. 

Phase  Record 

The  function  of  the  phase  record  is  to  provide  TRAWING  and  CNATRA  command 
personnel  with  information  about  the  student's  training  progress,  his  train- 
ing outcomes,  and  to  document  syllabus  status  achieved.  As  is  true  of  other 
records,  the  data  content  will  appear  under  several  descriptive  headings. 

These  are  described  below. 

Administrative.  This  type  of  information  will  consist  of  specific  stu- 
dent  identification  descriptors.  The  content  would  be  identical  to  that  used 
in  the  hop,  stage  and  administrative  records.  As  a minimum,  these  would 
include  (a)  the  student's  proper  name,  (b)  his  social  security  number,  (c) 
his  branch  of  service,  (d)  his  current  rank.  These  latter  two  data  elements 
are  not  essential  since  they  will  appear  in  the  Administrative  Record  which 
will  be  discussed  later  in  this  report. 

Phase.  This  type  of  information  will  consist  of  descriptors  that  document 
on  which  pipeline  syllabus  the  student  received  his  training,  the  D/M/Y  of 
the  beginning  of  training  and  the  D/M/Y  of  completion  of  phase  training. 

Where  pipeline  cross-overs  occur,  this  fact  should  be  reflected  under  this 
section  of  the  record. 
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PHASE  RECORD 


Ident i f icat ion 

• Name  of  Student 

• APL/SSN  of  Student 


Phase  Data 

• Phase  Name 

• D/M/Y  Started 

• D/M/Y  Completed 


Schedule  Position  Data 

• Projected  Completion  Date 

• Actual  Completion  Date 

• Cumulative  Hop  Days  lost: 

• Aircraft  Availability 

• Instructor  Availability 

• Student  Availability 
Other 


Cumulative  Grade  Data 

• FI i ght  Grade 

• Ground  School  Grade 

• Phase  Grade 


Flight  Hours  Data  - Cumulative 

• First  Pilot  - standard 

• Fi rst  Pilot  - actual 

• Copi lot  - standard 

• Copi lot  - actual 


Figure  5-9-  Phase  Record 
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Training  Status.  This  type  of  information  documents  what  actually  happened 
in  each  stage  but  does  it  in  a summary  form.  The  summary  data  will  determine, 
among  other  things:  (1)  the  syllabus  schedule  actually  achieved  by  the 

student;  (2)  the  factors  which  influenced  the  schedule  position  made  good; 

(3)  the  quality  level  of  the  student's  performance  in  the  ground  school  and 
flight  portions  of  the  syllabus;  and  (A)  the  total  flight  hours  required 
versus  the  standard  flight  hours  of  the  syllabus.  The  specific  data  elements 
which  this  section  should  contain  are  as  follows:  (l)  date  started  first  stage 
(D/M/Y):  (2)  date  completed  last  phase  stage  ( D/M/Y ) ; (3)  elapsed  days; 

(A)  syllabus  phase  schedule  position  expre.  in  relation  to  the  published 
'standard'  schedule  duration;  (5)  a tabulation  of  the  factors  which  contri- 
buted to  the  deviation  (if  any)  from  the  standard  schedule  (consider  express- 
ing these  as  both  absolute  numbers  and  in  terms  of  percentage,  i.e.,  what 
percentage  was  due  to  weather,  what  percentage  due  to  lack  of  aircraft  avail- 
ability, etc.);  and  (6)  the  number  of  solo  and  dual  flight  hours  flown 
expressed  in  hours  and  an  entry  which  compares  these  hours  to  the  number  in 
the  syllabus.  This  latter  number  would  have  to  be  computed  and  at  this  time 
it  is  suggested  that  a percentage  of  hours  flown  over  the  standard  or  below 
the  standard  be  computed. 

Phase  Outcome  Data.  In  the  above  section,  mention  was  made  of  two  com- 
puted values,  the  phase  schedule  position  and  the  flight  hours  position  in 
relation  to  the  published  standard.  The  raw  data  on  which  these  two  computed 
values  would  depend  would  appear  in  the  above  record  section  but  the  actual 
computed  values  would  appear  under  the  Phase  Outcome  Data.  Thus,  two  per- 
centage values  would  appear,  a percentage  above  or  behind  phase  schedule  and 
a percentage  less  than  or  greater  than  the  standard  flight  hours. 

One  other  type  of  Phase  Outcome  data  suggested  to  provide  a complete 
documentation  is  the  cumulative  grade  value  achieved  for  the  ground  school 
syllabus  and  the  cumulative  grade  value  for  the  flight  syllabus.  Considera- 
tion should  be  given  to  combining  these  two  values  to  secure  a 'final  phase 
grade'.  Any  one  of  the  previously  described  metrics  can  be  used  but  the  one 
that  provides  the  greatest  student  performance  discrimination  is  favored. 

The  administrative  utility  of  the  phase  grade  is  to  provide  some  quantitative 
expression  of  the  'quality  of  student  performance'  achieved  by  the  TRAWING 
and  its  supporting  squadrons. 

When  the  above  types  of  data  are  collected  and  computed  for  each  phase 
and  are  made  available  to  TRAWING  command  personnel,  they  would  be  in  a 
position  to  examine  their  own  operations  for  out-of-tolerance  conditions 
and,  where  any  exist,  review  the  options  available  to  them  and  execute  those 
that  seem  to  have  the  best  promise.  For  instance,  Pilot  Training  Rate  and 
Schedule  are  the  two  goals  which  each  TRAWING  is  trying  to  meet.  The  phase 
record  data  will  permit  them  to  identify  the  factors  which  influence  the 
pilot  training  rate  and  schedule  positions  and  also  to  relate  these  to  re- 
sources used  (number  of  aircraft  hours  flown,  number  of  flight  instructor 
hours  used,  number  behind  due  to  large  proportion  of  marginal  students,  etc.). 

The  above  types  of  relationships  can  be  made  available  to  command 
personnel  with  the  data  base  type  of  file  organization.  However,  before  this 
can  be  done,  the  command  personnel  must  indicate  that  they  wish  to  have  such 
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information  made  available  via  online  query  or  via  weekly  or  daily  reports, 
etc.  Weekly  summaries  can  be  fulfilled  by  batch  processing  whereas  daily 
reports  or  summaries  can  easily  be  provided  via  online  mode  of  operation. 

The  content  and  layout  of  the  phase  record  appears  in  Figure  5-9. 

Individualization  Information.  The  phase  record  as  described  so  far 
simply  tends  to  'document*  what  happened  during  a particular  phase  of  train- 
ing. But  how  can  this  record  contribute  to  greater  individualization  of 
tra i n i ng? 

The  phase  record,  together  with  the  individual  stage  records,  can  be  used 
to  secure  comprehensive  profile  information  on  each  student. 

As  the  student  nears  completion  of  the  last  stage  of  the  phase,  adminis- 
trative personnel  must  decide  whether  the  student  has,  in  fact,  met  all  of  the 
syllabus  requirements  and  whether  the  standard  of  performance  achieved  is 
sufficient  to  pass  the  phase  and  to  prepare  the  student  for  the  next  phase  of 
training.  A basic  thesis  of  the  SMS  development  effort  is  that  most  of  the 
traditional  grade  data  do  not  provide  a complete  enough  basis  for  the 
identification  of  student  weak  areas,  strong  areas  and  intermediate  perform- 
ance areas  of  the  flight  training  syllabus.  Therefore,  if  the  phase  summary 
record  is  to  play  a significant  role  in  decision-making  about  the  student, 
then  this  record,  together  with  the  information  contained  in  the  individual 
stage  records,  can  be  used  to  help  decide  on  the  proper  treatment  for  the 
student.  If  the  stage  records  were  used  to  detect  student  weak  areas 
systematically,  then,  by  the  time  the  student  completed  the  phase  his  known 
deficiencies  would  have  been  given  attention.  The  decision  to  be  made  at  the 
end  of  the  phase  is  whether  the  weak  areas  were,  in  fact,  detected,  what 
remediation  was  attempted  and  with  what  outcome.  With  this  information,  the 
TRAWING  commander  has  information  about  the  effectiveness  of  the  detection 
system,  the  effectiveness  of  remediation  efforts,  and  the  effectiveness  of 
efforts  to  achieve  and  maintain  the  projected  training  rate. 

Training  Summary  Record 

This  record  is  intended  to  document  major  factors  about  the  student's 
training  in  the  training  command.  The  major  factors  include:  (I)  those 

background  characteristics  of  the  student  that  have  shaped  his  entrance  skills 
and  knowledges;  (2)  the  specific  training  syllabus  under  which  the  student 
was,  in  fact,  trained;  (3)  the  outcome  of  the  training  either  in  terms  of 
attrition  or  completion  with  detailed  grade  and  qualifications  achieved;  and 
(1*)  administrative  events  such  as  student  disposition  board  occurrences, 
accident  data  and  other  administrative  decisions  that  impacted  on  the  stu- 
dent's career.  After  a tabulation  of  the  respective  record  sections  and 
their  specific  data  elements  is  made,  the  utility  of  the  sections/data 
elements  will  be  discussed. 

Student  Identification  Section.  This  section  would  contain  that  data 
which  CNATRA  has  established  as  a minimum  for  personnel  identification.  Shown 
in  this  section  are  the  student's  full  name,  social  security  number  and/or 
service  number,  the  branch  of  service  and  rank.  These  data  should  serve  to 
uniquely  identify  one  student  from  every  other  student. 


TRAINING  SUMMARY  RECORD 


Identi f icat ion 
Name  of  Student 
APL/SSN  Number 
Service  Branch 
Rank 

Background 

Procurement  Source 
FAR  Score 
AQT  Score 

Prior  Flight  Experience 

Training  Syllabus 
Pipeline 

Syllabus  Designations 
Phases  or  Stages 


Training  Outcome 
Completion  Date 
Attrition  Date 
Attrition  Stage 
Attrition  Hop/Maneuvers 
Attrition  Reason(s) 


Proficiency  Achieved 

• Cumulative  flight  grade 
' Ground  School  grade 

• Final  Overall  grade 

Resources  Used 

• Throughput  days 

• Solo  Flight  hours  - aircraft 

• Dual  Flight  hours  - aircraft 

Schedule  Position  Summary 

• Projected  completion  date 

• Actual  completion  date 

• Hop-days  lost: 

• Weather 

• Ai rcraft 

• Instructor 

• Student 

• Other 


Figure  5-10.  Training  Summary  Record 
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Background  Factors  Section.  Two  factors  of  the  student's  background 
which  shape  his  entrance  level  knowledges,  skills  and  attitudes  are  educa- 
tional experience  and  prior  flight  experience.  To  qualify  as  a candidate  for 
aviation  training  the  potential  candidate  must  take  several  tests.  The 
results  of  the  tests  are  recorded  as  scores  on  the  Flight  Aptitude  Record 
(FAR)  and  the  Aviation  Qualification  Test  (AQT).  A minimum  score  of  stanine 
3 and  5 on  these  respective  tests  has  generally  been  in  effect.  However,  the 
minimum  score  for  AQT  for  certain  branches  of  service  has  been  set  at  3.  This 
type  of  information  exists  in  some  administrative  documents  but  does  not  get 
recorded  on  the  student's  record.  This  section  would  be  a logical  place  for 
its  recording.  Whether  a student  naval  aviator  candidate  enters  from  civilian 
life  or  from  the  service  has  been  shown  to  influence  his  probability  of 
dropping  from  the  flight  program  voluntarily.  The  data  element  which  has 
made  such  determination  possible  is  the  identification  of  the  procurement 
source  of  the  student.  In  the  past,  a procurement  source  of  AOCS  has  been 
used  to  designate  a candidate  from  civilian  life.  NROTC  has  been  used  to 
designate  a candidate  who  was  enrolled  in  the  Navy  Reserve  Officers  Training 
Course . 

Prior  flight  experience  of  a candidate  could  have  been  via  the  Flight 
Indoctrination  Program  (FIP)  in  the  NROTC  program  or  through  a civilian  flight 
school.  Some  students,  who  were  released  for  the  convenience  of  the  govern- 
ment because  of  a surplus  of  students,  have  been  designated  as  COGS. 

Certainly,  this  previous  experience  will  impact  on  the  student's  rate  and 
level  of  progress.  When  making  analyses  of  groups  of  students,  it  is  essen- 
tial that  all  of  the  students  are  grouped  together  since  it  could  be  analagous 
to  comparing  apples  and  oranges.  This  error  can  only  be  avoided  if  the  stu- 
dent's record  contains  data  elements  that  make  these  important  identifications. 

Training  Syllabus  Section.  This  section  of  the  record  would  document 
in  which  of  the  pipeline  syllabuses  the  student  received  his  training  and  the 
specific  syllabus  codes  which  would  identify  the  specific  version  which  was  in 
effect  during  his  training. 

Syllabus  Change  Documentation.  At  this  point  it  is  desirable  to  amplify 
on  the  necessity  that  the  particular  syllabus  under  which  the  student  received 
his  training  be  documented.  For  instance,  in  the  period  prior  to  March  1970, 
student  naval  aviators  received  their  primary  flight  training  prior  to  their 
training  in  officer  candidate  school.  After  March  of  1970,  the  syllabus  was 
changed  to  flow  to  Aviation  Officer  Candidate  School  and  then  to  Primary 
Flight  Training.  Currently  under  consideration  and  about  to  be  implemented 
in  the  Basic  Phase  is  the  parallel  training  of  a student  in  Instrument  Flight 
and  Aerobatics  and  possibly  some  other  type  stage  training. 

Most  of  the  above  types  of  changes  are  made  and  recorded  at  an  admini- 
strative level  but  no  data  items  appear  in  the  student's  record  to  document 
this  type  of  syllabus  training.  Thus,  when  any  attempt  is  made  to  follow-up 
on  the  consequence  of  the  changes  the  data  elements  are  lacking  to  reveal 
which  students  were  trained  under  which  syllabus.  Therefore,  follow-up 
analyses  of  the  consequences  of  changes  in  syllabus  cannot  be  made  except  by 
combining  data  from  CNATRA  administrative  records  which  document  when  these 
changes  took  place,  what  the  nature  of  the  changes  was,  and  then  trying  to 
identify  those  students  who  were,  in  fact,  trained  accordingly.  Therefore, 
it  is  highly  recommended  that  CNATRA  establish  and  maintain  a master  syllabus 
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record  in  which  major  syllabus  changes  are  documented,  the  date  of  their 
implementation  and  the  syllabus  stages  which  are  effected.  Furthermore,  when- 
ever such  a change  occurs,  the  appropriate  syllabus  code  or  change  must  be 
recorded  on  the  student's  Training  Summary  Record.  To  accommodate  this  type 
of  data,  the  training  summary  record  concept  being  proposed  herein  contains 
a subtitle  under  which  changes  can  be  accommodated.  The  exact  data  elements 
that  would  go  into  this  section  depend  upon  the  system  CNATRA  establishes  for 
the  identification  of  syllabus  changes,  etc. 

Training  Status  Section.  An  examination  of  this  section  by  training  per- 
sonnel would  reveal  the  current  training  status  of  the  student.  The  data 
would  be  automatically  updated  from  data  which  becomes  available  to  the 
Phase  Record. 

The  current  training  status  of  the  student  would  include  the  following 
types  of  information: 

a.  current  stage  of  training 

b.  current  syllabus  hop 

c.  current  total  syllabus  schedule  position 

d.  factors  which  determined  student's  schedule  position 

e.  current  proficiency  status  - expressed  as  overall  grade  in 
standard  grade  - using  all  available  grades  assigned.  For 
deficiency  profile  data,  would  have  to  go  into  Phase  and 
Stage  records. 

f.  number  of  instructors  student  flew  with 

g.  total  number  of  solo  and  dual  hours  flown 

h.  probable  completion  date  based  on  current  squadrons  and  student's 
characteristics. 

Training  Outcome  Section.  This  section  would  contain  data  only  when  the 
student  has  completed  the  prescribed  syllabus  or  has  attrited.  For  those  who 
attrited  the  training  status  section  would  reveal  how  far  the  student  pro- 
gressed; data  elements  would  reveal  the  stage,  hop,  maneuver(s)  failed  and 
the  voting  outcome  of  the  board  and  admiral. 

For  those  students  who  completed  several  subsections  would  be  included. 
These  subsections  include  (a)  Proficiency  Achieved,  (b)  Schedule  Achieved, 

(c)  Flight  Hours  Required,  (d)  Flight  Instructors  Used,  (e)  special  qualifica- 
tions achieved  such  as  CQ,  gunnery,  etc.,  (f)  accident  data,  if  any,  and 
(g)  board  data  which  did  not  result  in  an  attrition. 

1.  Proficiency  Data:  All  students  achieve  a Final  Overall  Grade. 

This  final  overall  grade  is  currently  made  up  of  ground  school  grades  and 
flight  syllabus  grades.  When  these  two  are  combined  in  accordance  with  a 
formula  a single  grade  expressed  in  standard  score  form  is  secured.  The  actual 
value  of  this  score  will  permit  any  student  to  be  compared  with  any  other  stu- 
dent and  a ranking  of  students  can  be  made  where  administrative  decisions  need 
such  information. 

Although  the  single  grade  value  permits  a comparison  of  students  with  each 
other,  it  does  not  reveal  the  existence  of  any  strong  or  weak  flight  maneuver 
skills.  To  secure  these  types  of  data,  it  would  be  necessary  to  access  the 


stage  records  of  the  student.  Strong  and  weak  skill  areas  tend  to  be  specific 
to  the  aircraft  and  stage  syllabus  maneuvers.  The  areas  of  strengths  and 
weaknesses  can  be  readily  identified  by  the  performance  reliability  metric 
and/or  profile.  This  topic  was  discussed  under  the  Stage  Record  description. 


2.  Schedule  Achieved:  This  section  would  contain  information  and 

data  elements  which  revealed  the  actual  syllabus  schedule  achieved.  The 
information  would  appear  as  (a)  standard  syllabus  days/weeks,  (b)  achieved 
syllabus  days/weeks,  (c)  percentage  of  days/weeks  ahead  or  behind  syllabus 
standard,  (d)  tabulation  of  actual  number  of  days  lost  due  to  weather,  air- 
craft availability,  instructor  availability  and  student  availability.  These 
data  could  be  supplemented  by  computational  data  which  tabulate  the  percentage 
of  time  lost  to  each  of  these  factors;  or  this  type  of  computation  may  be 
included  in  an  application  program  that  retrieves  the  raw  schedule  data  and 
then  makes  the  percentage  computation,  especially  for  a large  group  of 
students.  For  instance,  weather  factors  could  conceivably  have  a different 
effect  on  schedule  based  on  the  season  of  the  year.  To  perform  a relevant 
analysis  on  this  differential  effect  it  would  be  necessary  to  look  at  how 
weather  influenced  each  stage  type,  but  with  the  precaution  that  the  same  time 
periods  would  be  examined.  How  would  those  stages  be  identified,  say,  that 
received  their  training  in  the  spring  of  the  year?  This  would  have  to  be  done 
by  looking  at  those  students  who  began  their  stages  in  the  month  of  March  and 
completed  by  May.  As  described  in  the  proposed  Stage  Record  the  data  elements 
would  be  there  and  such  an  analysis  would  be  facilitated. 

3.  Flight  Hours  Required:  Each  student  flies  a certain  number  of 

hops  and  when  the  durations  are  summated  a total  flight  hours  value  is 
available.  Each  squadron  currently  records  the  total  hours  of  solo  and  dual 
for  each  stage.  The  phase  record  would  summarize  the  stage  record  data.  When 
the  student  has  completed  his  training  the  total  that  appears  on  the  phase 
record  would  automatically  be  transferred  to  the  summary  record  and  be  placed 
in  this  section. 

In  addition  to  containing  the  total  required  hours  the  standard  number  of 
hours  required  for  the  particular  syllabus  would  appear.  Thus,  the  number  of 
hours  above  or  below  the  standard  could  readily  be  seen  when  the  record  was 
examined.  An  optional  data  element  would  be  to  compute  a percent  hours  above 
or  below  the  standard  and  the  placement  of  this  value  in  this  section. 

The  utility  of  these  types  of  data  would  support  the  examination  of  any 
student's  progress  in  terms  of  the  number  of  hours  needed  to  qualify  as  a naval 
aviator.  An  operational  definition  of  a good  student  pilot  could  well  include 
the  number  of  flight  hours  needed  to  qualify.  There  is  already  preliminary 
evidence  that  higher  graded  students  are  given  less  hours  by  their  flight 
instructors  than  lower  graded  students.  Thus,  flight  hour  ranking  among  stu- 
dents may  well  be  a useful  metric  that  can  be  used  to  differentiate  between 
students  who  have  the  same  overall  cumulative  training  grade.  The  analysis  of 
flight  hour  distributions  of  students  at  the  end  of  individual  stages  and  phases 
reveals  more  diversity  than  grade  data  distribution.  Admittedly,  this  is  a 
secondary  type  of  evaluation  criterion  in  comparison  to  the  grade  value  per  se. 

The  content  and  layout  of  the  summary  record  is  shown  in  Figure  5.10. 
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Admin i s t rative  Reco rd 


This  record  would  contain  data  pertaining  to  the  student's  Navy  career 
per  se  rather  than  any  data  about  his  training.  No  attempt  will  be  made  to 
itemize  these  data  since  they  are  dictated  by  established  policy  and  proce- 
dures. All  of  the  records  already  described  would  remain  within  the  training 
command  and  would  not  be  forwarded  when  the  student  left  for  operational 
training.  However,  some  liaison  with  the  operational  training  command 
should  be  retained  to  determine  the  utility  of  forwarding  some  types  of 
student  data  which  would  reflect  the  student's  various  proficiency  levels 
achieved.  If  the  operational  training  command  decides  to  maximize  individ- 
ualized training  treatments  then  some  data  about  the  student  may  be  required. 


SECTION  VI 
SUPPORT  TASKS 


INTRODUCTION 

The  development  plan  and  structure  contained  a third  major  task.  This 
third  task  is  entitled  "Support  Tasks."  The  idea  behind  this  was  to  assemble 
data  and  information  that  could  impact  on  the  design  of  the  Data  Management 
System  and  on  the  Student  Management  System.  Four  such  support  tasks  are 
shown  in  Figure  3-3,  the  task  and  phase  summary.  The  scope  of  effort  and  the 
results  of  each  of  these  areas  are  reviewed  below. 


EXPERIMENTAL  DATA  BANK 


The  Experimental  Data  Bank  is  a computerized  file  that  contains  selected 
records  of  student  naval  aviators.  These  records  cover  students  who  received 
and  completed  their  training  during  the  1970/71  calendar  years.  The  records 
were  placed  on  magnetic  tape  files  and  used  the  UNIVAC  418  computer  for  their 
preparation  and  analysis. 

The  primary  reason  for  the  creation  of  the  Experimental  Data  Bank  was  to 
provide  the  means  by  which  objective,  empirical,  quantitative  information 
could  be  generated  about  a wide  range  of  topics.  It  supports  all  of  the 
quantitative  data  developed  for  the  Student  Management  System.  It  also  pro- 
vides all  of  the  quantitative  data  which  supports  the  contributions  made  to 
the  Student  Success  Prediction  System  and  to  the  Training  Systems  Analysis  task 
areas  described  below.  Thus,  the  creation  and  operation  of  the  Experimental 
Data  Bank  formed  a cornerstone  upon  which  the  development  program  was  built. 


The  broad  types  of  data  contained  in  this  bank  are  described  in  the 
following  paragraphs. 

1 . FI iqht  Maneuver  Error  Data 

For  each  hop  flown  with  a flight  instructor  the  errors  made  by  the 
student  are  recorded  on  a standard  form.  This  form  contains  the  list  of  all 
maneuvers  flown,  the  grade  made  on  each  maneuver,  and  some  written  comments 
about  the  nature  of  the  errors  the  student  made  on  those  maneuvers  which  were 
graded  below  average  and  average.  For  a group  of  25  students  who  had  com- 
pleted their  training  of  the  presolo  stage  of  training,  the  hop  record  forms 
were  secured.  To  enter  the  error  data  into  the  computer  a coding  system  was 
devised  to  identify  the  specific  maneuver  and  maneuver  sub-task  to  which  the 
errors  belonged. 

Within  a training  system  that  uses  subjective  grade  assignments  such  as 
average,  below  average,  etc.,  the  error-data  may  be  viewed  as  being  less 
influenced  by  instructor  biases.  A grade  is  a subjective  expression  of  the 
errors  made  by  the  student.  The  value  of  the  error-data  is  that  it  is  these 
errors,  whether  they  be  constant  or  random,  that  must  form  the  basis  of  any 
meaningful  student  remediation  effort  or  of  a program  to  implement  greater 
student  individualization.  It  is  the  errors  that  must  be  remediated  and  not 
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the  grade  which  is  but  a symptom  of  the  errors.  A student  maneuver  error 
profile  was  developed  to  help  give  quantitative  tabulations  of  these  errors. 

A sample  profile  is  shown  as  Figure  3.7  in  reference  I.  The  relation  of 
error-data  to  grade  data  is  discussed  in  detail  on  page  3-5  of  reference  2. 

A complete  discussion  of  these  data  appears  on  pages  4-2  through  4-10  of 
reference  2. 

2.  Flight  Maneuver  Grade  Data 

For  a selected  group  of  students,  the  grades  made  on  each  maneuver  of 
each  hop  of  the  three  phases  of  training  were  placed  into  the  data  bank.  This 
amounted  to  approximately  3,000  maneuver  grades  per  student,  plus  a separate 
hop  grade  for  each  of  approximately  250  hops. 

The  data  were  analyzed  systematically  to  develop  student  stage  grade 
profiles  for  each  stage  in  the  basic  phase  and  each  stage  in  the  advanced 
phase  of  training.  These  profiles  are  contained  in  references  3 and  4 
respectively.  The  data  were  also  used  to  secure  statistics  on  student 
throughput  time  and  flight-hours  required  for  students  in  each  of  the  stages 
as  well  as  for  each  of  the  two  phases  of  flight  training.  These  data  are 
discussed  for  the  basic  phase  in  reference  3 and  for  the  advanced  phase  in 
reference  4. 

3.  Student  Evaluation  Board  Data 

When  a student  is  having  difficulty  with  the  training  syllabus,  a 
board  of  officers  convenes.  This  board  reviews  the  student's  total  situation; 
his  previous  performance,  his  current  performance  and  any  personal  problems 
that  he  may  be  experiencing.  Based  on  their  evaluation,  the  board  recommends 
either  that  the  student  be  dropped  from  further  training  or  retained  with  a 
recommendation  of  the  type  of  remedial  training  he  should  be  assigned.  Under 
this  system,  some  students  may  accumulate  up  to  three  or  four  board  hearings 
while  they  complete  the  total  training  program. 

Previous  research  conducted  by  the  Aerospace  Medical  Research  Laboratory 
has  revealed  that  the  use  of  a board  event  on  students  contributed  to  an 
improved  prediction  of  their  future  risk  of  pass  or  fail.  Thus,  there  was 
interest  in  placing  as  much  of  these  types  of  data  into  the  data  bank  as 
possible  to  help  exploit  the  data  on  a large  group  of  students. 

When  the  above  types  of  data  are  combined  with  the  selection  and  training 
data  which  reside  on  the  NAMRL's  tape  files,  they  represent  the  consolidation 
of  all  available  data  on  student  naval  aviators  for  those  training  years. 
Although  Rowland  6 Company  personnel  used  these  data  as  described  above,  they 
have  not  been  completely  exploited.  This  exploitation  was  intended  to  be  done 
by  the  Aerospace  Medical  Research  Laboratory  research  personnel. 

TRAINING  SYSTEM  ANALYSIS 

The  above  data  which  were  placed  into  the  Experimental  Data  Bank  offered 
an  opportunity  for  examination  in  certain  combinations  so  as  to  provide 
quantitative  information  about  certain  features  of  the  training  system  from 
which  they  were  secured.  These  features  included  the  following:  (1)  student 

attrition  statistics;  (2)  student  stage  graduate  quality  profiles; 
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(3)  student  stage  flight-hour  distributions;  (4)  student  throughput-days  to 
complete  individual  stages  and  phases  of  their  flight  training;  and  (5)  student 
quality  control  mechanisms.  Each  of  these  features  are  summarized  below. 

Student  Attrition  Statistics 


Of  what  value  are  statistics  on  student  attrition?  The  primary  value  is 
that  these  data  identify  the  stage,  the  hop  and  the  specific  syllabus  maneuver 
where  either  a large  group  of  students  lacks  the  aptitudes  and  skills  to 
handle  and  therefore  fails,  or  where  the  training  system  is  ineffective  in 
adapting  its  instructional  techniques  to  the  training  needs  of  the  students. 

Any  attempt  to  cope  with  the  student  attrition  situation  must  come  to  grips 
with  these  syllabus  situations  as  revealed  by  the  statistics.  These  statistics 
appear  in  the  first  three  annual  reports  (references  1,  2,  and  3)  and  no  attempt 
will  be  made  to  tabulate  them  here. 

Student  Stage  Graduate  Quality  Profiles 

Although  the  flight  attrition  statistics  reveal  information  about  those 
students  who  fail  the  syllabus,  grade  profiles  of  student  stage  graduates 
demonstrate  the  performance  or  skill  level  achieved  by  those  who  are  graduated 
by  the  training  system.  These  profiles  can  reveal  those  stage  maneuvers  where 
students  have  the  lowest,  average,  and  the  highest  proficiency. 

Again,  those  who  are  concerned  with  improving  either  the  quality  of  the 
student  graduates  or  the  quality  of  the  training  system  effectiveness  would 
find  these  profiles  to  be  a basic  point  of  departure.  These  are  presented  in 
references  2,  3 and  4. 


Student  Stage  Flight-Hour  Distributions 

Students  enter  the  training  system  with  wide  ranges  in  aptitudes.  Through- 
out their  training,  they  reveal  continued  diversity  on  nearly  all  available 
measures.  This  diversity  was  found  to  exist  on  the  number  of  flight  hours 
each  student  was  given  so  as  to  qualify  them  for  stage  proficiency.  Flight 
hours  data  are  reported  in  references  3 and  4. 


An  interpretation  of  these  data  was  offered  on  page  3~ 29  of  reference  3- 
The  data  were  interpreted  as  evidence  for  the  existence  of  a 'differential 
student  learning  rate  of  flight  skills.'  It  could  also  be  interpreted  to 
mean  that  flight  instructors  adjusted  the  hop  duration  to  the  needs  of  the 
student.  The  students  with  the  lower  grades  were  given  longer  flight  hops 
than  students  in  either  the  medium  or  high  grade  groups. 

Throughput  Days 

One  possible  measure  of  the  effectiveness  of  a training  system  is  the 
number  of  days  it  requires  to  train  an  individual  student.  For  a group  of 
students  this  may  mean  an  average  of  throughput  days. 


The  student  data  placed  into  the  Experimental  Data  Bank  offered  an  oppor- 
tunity to  collect  statistics  on  student  throughput  time.  Computations  were 
made  on  one  flight-stage  in  the  advanced  phase  of  training  and  also  on  the 
total  advanced  phase.  These  data  are  reported  in  the  Phase  IV  annual 
report  (4). 
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The  point  made  in  this  report  is  that  such  computations  can  be  secured 
from  existing  student  record  data  and  that  the  use  of  throughput  days  should 
be  of  interest  to  management  level  administrators  who  are  concerned  with 
measures  of  training  system  effectiveness  and  with  means  for  improving 
existing  training  systems.  Knowing  that  there  is  a large  diversity  among 
student  throughput  days  may  be  one  piece  of  interesting  information.  What  is 
more  important  to  pursue  are  the  reasons  for  the  diversity.  This  topic  was 
covered  in  considerable  detail  in  the  record  concept  and  content  of  the  Stu- 
dent Management  System  (Section  V of  this  report). 

Quality  Control 

The  grade  system  is  used  to  help  achieve  quality  control  of  the  students 
who  pass  each  flight  stage  in  their  training.  An  examination  of  the  student 
flight  attrition  statistics  reveals  that  a very  very  small  percentage  are 
failed  due  to  low  flight  grades  per  se.  The  great  majority  are  failed  by  the 
‘down1  system.  It  would  appear,  therefore,  that  at  the  low  end  of  the  student 
grade  scale  the  down  system  is  the  most  used  technique  to  assure  a minimum 
acceptable  student  quality  level.  This  latter  finding  was  reported  and  dis- 
cussed in  reference  2. 

A corollary  observation  was  also  reported  in  that  reference.  It 
observed  that  a student  naval  aviator  who  flew  approximately  220  hops  may 
have  been  instructed  and  evaluated  by  as  many  as  90  separate  flight  instructors 
by  the  time  he  completed  the  operational  phase  of  training.  One  can  interpret 
these  data  to  mean  that  the  student  was  man-rated  repeatedly  for  his  satis- 
factory performance  by  each  of  the  90  instructors.  This  repeated  man-rated 
system  has  probably  contributed  significantly  to  the  knowledge  and  belief  that 
the  final  student  product  can  be  expected  to  perform  both  satisfactorily  and 
rel  iabl y (2 , p. A-Al ) . 


* STUDENT  SUCCESS  PREDICTION 

» 

A third  subject  area  of  the  support  tasks  is  student  success  prediction. 

It  will  be  recalled  that  the  Naval  Aerospace  Medical  Research  Laboratory 
has  a support  role  to  the  Chief  of  Naval  Air  Training  to  provide  prediction 
« estimates  on  students  who  are  in  the  training  program.  This  includes  students 

g who  are  being  trained  as  naval  aviators  and  those  who  are  being  trained  for 

naval  flight  officer  billets.  The  naval  flight  officers  serve  as  radar 
’ intercept  officers  on  fighter  type  aircraft,  as  navigator-equipment  operators 

on  reconnaissance  aircraft,  as  Tactical  Command  officers  aboard  anti- 
submarine aircraft,  etc. 

The  Student  Success  Prediction  System  was  developed  and  is  operated  by 
NAMRL.  NAMRL  provides  scores  to  squadron  training  personnel  on  students  whom 
the  squadron  wishes  to  review  because  these  students  are  encountering  some 
di f f i cul ty  wi th  the  training  syllabus.  The  scores  reveal  the  students' 
chances  of  completing  the  total  syllabus  successfully.  Such  scores  are 
computed  for  about  2k  syllabus  places  in  the  primary  and  basic  phases  of  the 
training  sy 1 1 abus . 


The  actual  value  of  the  score  is  influenced  by  various  factors.  These 
include  the  grades  a student  made  on  a battery  of  selection  tests,  grades 


made  on  entrance  tests  into  officer  candidate  training,  grades  made  on  academic 
subjects  taken  in  the  officer  training,  and  by  grades  made  in  the  ground  school 
and  flight  syllabus  of  the  flight  training  syllabus. 

The  thrust  of  the  Rowland  6 Company  effort  was  to  identify  new  kinds  of 
existing  but  not  being  used  student  aptitude  and  training  performance  data. 

In  addition,  a considerable  amount  of  effort  was  devoted  to  the  identification 
of  alternative  ways  of  computing  flight  proficiency  grades  which  could  be  used 
in  the  development  of  predictive  equations.  The  results  of  these  efforts  are 
reported  in  references  3 and  4. 

The  results  reported  contained:  (1)  new  aptitude  measures;  (2)  the  use  of 

cumulative  stage  hop  grade  values;  (3)  the  use  of  a grade  that  was  more  sensi- 
tive to  the  flight  syllabus  maneuvers  on  which  students  actually  failed;  (4) 
the  use  of  a grade  that  discriminated  more  levels  of  maneuver  skills  among 
students  than  the  current  grade  computation  method;  and  (5)  the  use  of  a 
student  flight  performance  reliability  (consistency)  score.  Some  55  places 
in  the  total  training  syllabus  were  identified  where  these  other  performance 
measures  could  be  applied  and  tested  for  their  predictive  utility  (12). 

The  student  data  from  which  all  of  these  measures  could  be  secured  were 
placed  into  the  Experimental  Data  Bank  and  the  taped  files  delivered  to  the 
custome  r . 
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